Skip to content

YARN-11429. Improve the updateTestDataAutomatically in TestRMWebServices.#5736

Merged
brumi1024 merged 1 commit intoapache:trunkfrom
tomicooler:YARN-11429
Jun 14, 2023
Merged

YARN-11429. Improve the updateTestDataAutomatically in TestRMWebServices.#5736
brumi1024 merged 1 commit intoapache:trunkfrom
tomicooler:YARN-11429

Conversation

@tomicooler
Copy link
Contributor

@tomicooler tomicooler commented Jun 12, 2023

Description of PR

The goal is to have the same json formatting options when using the updateTestDataAutomatically helper function.

How was this patch tested?

I verified that the updated JSON content is the same as before:

# before
➜  /Users/tdomok/Work/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/webapp git:(YARN-11429) for f in *.json; do echo -n "$f "; cat $f | jq | md5sum; done
scheduler-response-AbsoluteMode.json 3cd84395f2864bbfcab55d1f81be5f44  -
scheduler-response-AbsoluteModeLegacyAutoCreation.json 4dbd90be628a52fa16eac0bde3774d6d  -
scheduler-response-PerUserResources.json eda777af4bf2f6cbfd3dc4b1a0ac945a  -
scheduler-response-PercentageMode.json 9a1dbd77cb034390092a553bfb04c078  -
scheduler-response-PercentageModeLegacyAutoCreation.json 9ca117ab46a4eb8be261650d26b17ae4  -
scheduler-response-WeightMode.json b87bc438cc66619e16edec4acef1dafb  -
scheduler-response-WeightModeWithAutoCreatedQueues-After.json 4265fb1c65813a44989f9f77cbd5e862  -
scheduler-response-WeightModeWithAutoCreatedQueues-Before.json a239664f30b71478991b0504a66dc4a5  -
scheduler-response.json 6e5b607d133ddf4e00eda23842b274db  -

# after
➜  /Users/tdomok/Work/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/webapp git:(YARN-11429) for f in *.json; do echo -n "$f "; cat $f | jq | md5sum; done
scheduler-response-AbsoluteMode.json 3cd84395f2864bbfcab55d1f81be5f44  -
scheduler-response-AbsoluteModeLegacyAutoCreation.json 4dbd90be628a52fa16eac0bde3774d6d  -
scheduler-response-PerUserResources.json eda777af4bf2f6cbfd3dc4b1a0ac945a  -
scheduler-response-PercentageMode.json 9a1dbd77cb034390092a553bfb04c078  -
scheduler-response-PercentageModeLegacyAutoCreation.json 9ca117ab46a4eb8be261650d26b17ae4  -
scheduler-response-WeightMode.json b87bc438cc66619e16edec4acef1dafb  -
scheduler-response-WeightModeWithAutoCreatedQueues-After.json 4265fb1c65813a44989f9f77cbd5e862  -
scheduler-response-WeightModeWithAutoCreatedQueues-Before.json a239664f30b71478991b0504a66dc4a5  -
scheduler-response.json 6e5b607d133ddf4e00eda23842b274db  -

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'YARN-11429. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@tomicooler
YARN-11429. Improve the updateTestDataAutomatically in TestRMWebServi…
e92a9dd 
…ces.

Change-Id: I1a26bdfa52deebe56578b04fc03591a2e2896f10
@hadoop-yetus
Copy link

hadoop-yetus commented Jun 12, 2023

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 57s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 jsonlint 0m 0s jsonlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 10 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 45m 26s trunk passed
+1 💚 compile 1m 7s trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚 compile 1m 0s trunk passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu120.04-b09
+1 💚 checkstyle 1m 2s trunk passed
+1 💚 mvnsite 1m 10s trunk passed
+1 💚 javadoc 1m 7s trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚 javadoc 0m 54s trunk passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu120.04-b09
+1 💚 spotbugs 2m 12s trunk passed
+1 💚 shadedclient 23m 11s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 53s the patch passed
+1 💚 compile 0m 56s the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚 javac 0m 56s the patch passed
+1 💚 compile 0m 50s the patch passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu120.04-b09
+1 💚 javac 0m 50s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 43s the patch passed
+1 💚 mvnsite 0m 51s the patch passed
+1 💚 javadoc 0m 45s the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚 javadoc 0m 41s the patch passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu120.04-b09
+1 💚 spotbugs 1m 58s the patch passed
+1 💚 shadedclient 22m 31s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 118m 48s hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 43s The patch does not generate ASF License warnings.
229m 52s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5736/1/artifact/out/Dockerfile
GITHUB PR #5736
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets jsonlint
uname Linux 3a552020f5c1 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / e92a9dd
Default Java Private Build-1.8.0_362-8u372-gaus1-0ubuntu120.04-b09
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-gaus1-0ubuntu120.04-b09
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5736/1/testReport/
Max. process+thread count 935 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5736/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@brumi1024
Copy link
Member

brumi1024 commented Jun 14, 2023

Thanks @tomicooler for the update, looks good to me, and improves the readability.

It's not strictly related to this patch, and probably not in its scope, but I don't really get why are we doing the following flow for a simple JSON pretty print: string -> JSON POJO (lib#1) -> string -> create an ObjectMapper per pretty print (it's a really heavyweight object, it should be reused) -> JSON POJO (lib#2) -> string again. This is much-much slower than necessary. Probably this should be refactored later.

Regardless I'll merge this now, and create a followup ticket.

@brumi1024 brumi1024 merged commit f4b70b3 into apache:trunk Jun 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@tomicooler @hadoop-yetus @brumi1024